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REMARKS 

Claims 1-44 are pending, with claims 1, 14, 24, 32, 40, and 
42 being independent. Claims 24, 27, 28, 32, 35, 36, 42, 43, 
and 44 have been cancelled by this amendment without prejudice. 
Claims 2, 3, 12, 18, 22, 25, 29, 30, 33, 37, 38, and 41 have 
been amended. New claims 45-4 9 have been added. No new matter 
has been added. Reconsideration and allowance of the above - 
referenced application are respectfully requested. 

Objections to the Specification: 

The title stands objected to as allegedly failing to be 
descriptive. This objection is respectfully traversed. The 
original title is in fact descriptive of the disclosed subject 
matter generally, and there is no requirement that the title 
directly correspond to the claims; the claims themselves define 
the invention, not the title. Nonetheless, the title has been 
amended to be more clearly indicative of the disclosed and 
claimed subject matter. Thus, withdrawal of the objection to 
the title is respectfully requested. 

The Background and Summary sections stand objected to for 
allegedly failing to comply with the suggested application 
format as outlined in MPEP §§ 608.01(c) & 608.01(d). These 
objections are respectfully traversed. The present Background 
section includes a statement of the field of art to which the 
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invention pertains in paragraph [0003] , and also includes a 
description of the related art in paragraph [0004] . The Summary 
section includes a general statement of the invention in 
paragraphs [0005] - [0007] . 

Additionally, the cited portions of the MPEP, and 37 CFR 
1.73, make clear that the rule merely states what a 
specification should include, and falls short of making the 
described application format mandatory. Since compliance with 
Rule 73 is entirely voluntary , it is respectfully suggested that 
the application is totally complete as filed, and therefore the 
requirements for changes to the Background and Summary sections 
are respectfully traversed. Thus, withdrawal of the objections 
to the Background and Summary sections is respectfully 
requested. 

The .disclosure stands objected to for various 
informalities. The specification has been amended to correct 
these informalities. Thus, withdrawal of the objection to the 
disclosure is respectfully requested. 

Claim Rejections under 35 U.S.C. 112: 

Claims 1-44 stand rejected under 35 U.S.C. 112, first 
paragraph, as allegedly not being enabled with respect to "at 
least one image sensor". The office action asserts, "on pages 
24-25, paragraphs [0064] and [0065], [the specification states] 
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that a stereo camera is necessary and this is illustrated in 
Fig. 2 for a mobile image sensor. " This contention is 
respectfully traversed. 

Paragraph [0064] states, "The system 200 can be a self- 
contained portable tracking package (e.g., a backpack) that 
includes a stereo camera 210." (Emphasis added.) Neither this 
portion of the specification, nor any other portion of the 
specification suggests a requirement that an image sensor being 
tracked in the environment be a stereo camera. Paragraphs 
[0032] and [0061] make very clear that only a single image 
sensor need be tracked in the environment. Furthermore, 
paragraph [0065] makes clear that even if a stereo camera is 
used, only one channel need be used for video projection. In 
addition, FIGS. 6, 8, and 10 clearly show that only a single 
projection frustum is used, for each of one or more tracked 
image sensors in the environment, in the image projection 
example being described. Therefore, one skilled in art would 
clearly understand from the specification that only a single 
image sensor need be tracked in the environment. Thus, 
withdrawal of the 35 U.S.C. 112, first paragraph, rejection of 
claims 1-44 is respectfully requested. 

Claims 1-44 stand rejected under 35 U.S.C. 112, second 
paragraph, as allegedly being indefinite. This contention is 
respectfully traversed. With respect to independent claims 1, 
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14, 24, 32, 40, and 42, the above remarks are equally 
applicable. In addition, it is noted that none of claims recite 
a "camera", as is suggested in the office action. These claims 
recite "at least one image sensor" , of which a camera is but one 
example. One skilled in the art would clearly understand the 
subject matter being claimed in view of the specification. 
Thus, these claims are definite as written. 

With respect to claims 2 8 and 36, the term "morphological" 
is definite in view of the knowledge of one skilled in the art. 
Nonetheless, the term "morphological" has been removed in the 
rewriting of claims 2 9 and 3 7 into independent form, and claims 
28 and 36 have been cancelled without prejudice. Thus, this 
rejection of claims 2 8 and 3 6 has been obviated by the present 
amendment . 

In view of the above, withdrawal of the 35 U.S.C. 112, 
second paragraph, rejection of claims 1-44 is respectfully 
requested. 

Allowable Subject Matter: 

Claims 12, 22, 29, and 37 have been indicated as allowable 
if rewritten in independent form, and if the rejection (s) under 
35 U.S.C. 112, second paragraph, are overcome. Claims 12, 22, 
29, and 37 have been rewritten in independent form. In view of 
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this amendment, and the remarks above, allowance of these claims 
is respectfully requested. 

Claim Rejections under 35 U.S.C. 103: 

Claims 1-2, 5, 7-9, 11, 14, 21, and 40 stand rejected under 
35 U.S.C. 103(a) as allegedly being unpatentable over Kumar et 
al (US PGPub 2001/0038718 Al) in view of Haala et al (Generation 
of 3D City Models From Airborne Laser Scanning Data) . Claims 3, 
18, and 41 stand rejected under 35 U.S.C. 103(a) as allegedly 
being unpatentable over Kumar in view of Haala as applied to 
claim 2 above, and further in view of You et al (Automatic 
Object Modeling for 3D Virtual Environment) . Claims 4, 10, 15- 
17, and 20 stand rejected under 35 U.S.C. 103(a) as allegedly 
being unpatentable over Kumar in view of Haala as applied to 
claim 2 above, and further in view of Arpa et al (US 
2003/0085992 Al) . Claims 6 and 19 stand rejected under 35 
U.S.C. 103(a) as allegedly being unpatentable over Kumar in view 
of Haala as applied to claim 1 above, and further in view of 
Weinhause et al (Texture Mapping 3D Models of Real -World 
Scenes). Claims 13 and 23 stand rejected under 35 U.S.C. 103(a) 
as allegedly being unpatentable over Kumar in view of Haala as 
applied to claim 1 above, and further in view of Pryor et al (US 
2004/0046736 Al) . These contentions are respectfully traversed. 
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Kumar describes the alignment of input images with 
reference images that are already aligned with data. (See Kumar 
at Abstract.) In Kumar, alignment is performed in several ways, 
including the use of a "projective transf ormation" , which 
provides a mathematical relationship that describes the mapping 
from one two dimensional (2D) image to another. (See e.g., 
Kumar at paragraphs [0034], [0035], and [0037].) Kumar further 
makes use of the term "projection" as a mapping from three 
dimensional (3D) points (vertices of a 3D mesh) onto a 2D image 
plane for producing new reference images. (See e.g., Kumar at 
paragraph [0103].) In both cases the term "projection" is used, 
but does not describe the subject matter of the present claims. 

Independent claims 1 and 4 0 recite, "projecting real-time 
video imagery information from the at least one image sensor 
onto the three dimensional model based on the tracked 
orientation information". Independent claim 40 recites, "a 
dynamic fusion imagery projection component that projects real- 
time video imagery information from at least one image sensor 
onto the three dimensional model based on orientation 
information of the at least one image sensor tracked in the 
environment with respect to the three dimensional model in real- 
time" . In view of the specification, this claim language 
clearly indicates a mapping from 2D image data onto a set of 3D 
geometric surfaces that can be viewed from arbitrary viewpoints. 
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This is analogous to using a slide projector to project an image 
onto the walls and objects in a room. The process and effect of 
such a projection is entirely different than the single mapping 
or "projective transformation", which relates a 2D image to 
another 2D image, as described in Kumar. There is no reference 
in Kumar, under any terminology, to a process of image 
projection, as is presently claimed. 

In addition, there is no motivation to combine the 
references. Kumar is not directed to generation of three 
dimensional terrain maps, as suggested in the official action. 
Rather, Kumar is directed to aligning 2D images from a sensor 
(e.g., dynamically generated video from an aerial platform) with 
reference imagery, and then overlaying data associated with the 
reference imagery on the 2D images. Each of independent claims 
1, 14, and 40 require that "the at least one image sensor" is 
tracked "in the environment", which allows the claimed image 
projection onto the three dimensional model from the perspective 
of the image sensor. In contrast, Kumar teaches away from the 
proposed combination in that Kumar makes clear the "two- 
dimensional projective transform [...] is valid when there is only 
a small view-point difference between the rendered reference 
image and the video frame and the distance between camera to 
ground is large as compared to the height of objects in the 
scene." (See Kumar at paragraph [0084].) Thus, there is no 
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motivation to combine Kumar, Haala, and Arpa as proposed, and 
even if such combinations could be made, they would not result 
in the presently claimed subject matter. 

Thus, independent claims 1, 14, and 4 0 should each be 
allowable. Dependent claims 4-11, 13, 15-17, 19-21, and 23 each 
depend from an allowable base claim for at least the reasons 
discussed above and should thus also be allowable. 

In addition, claims 2, 18, and 41 have been amended to be 
in independent form and to recite additional aspects of the 
three dimensional model generation. Dependent claim 3 has been 
amended to recite additional aspects of the selection of 
geometric primitives and the parametric fitting, and new 
dependent claim 45 recites still further details. New dependent 
claims 46-49 recite similar subject matter as claims 3 and 45. 
Support for these amendments can be found in paragraphs [0038] 
to [0059] of the specification. 

The claimed subject matter allows a user to define part of 
a building, rather than the whole structure at once. Users may 
select one primitive at a time to describe parts of an arbitrary 
structure, which allows the modeling approach to readily scale 
to arbitrary building complexity. This human-guided approach 
can result in better models than the completely automated method 
of Haala, but without requiring inordinate user input. As 
stated in the specification, "A person can assist in selecting 
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the group type and thus the associated primitives [...] and this 
scene segmentation and model fitting approach to building a 3D 
model of a large urban environment can enable rapid creation of 
models of large environments with minimal effort and low cost." 
(See specification at paragraphs [0058] and [0059].) 

In addition, using the constrained best fitting of 
geometric primitives approach can result in improved 3D models 
that can be rapidly developed even if limited input data is 
available, such as might be the case in an urban, battlefield 
environment. The use of constrained 3D shapes, such as spheres 
and cuboids, allows rapid modeling of buildings even when the 
input data is less than ideal. As described in the 
specification, "once the buildings are detected and segmented, 
the predefined geometric primitives can be iteratively fitted to 
the data and the best fitting models can be used to represent 
the building structures. [...] Moreover, undersampled building 
details and occlusions from landscaping and overhangs can also 
be handled without resulting in data voids." (See specification 
at paragraphs [0058] and [0059].) 

In contrast, Haala performs plane fitting, which makes it 
difficult to segment the edges of buildings. "Due to the very 
pretentious problem of automatic interpretation of height or 
range data, optimal results can only be achieved by using 
supplementary sources of information. Within the approach 
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presented in this paper the segmentation of planar surfaces from 
the DSM [Digital Surface Model] , which is a prerequisite for the 
3D reconstruction of the buildings, is supported by given ground, 
plans." (See Haala at Abstract.) The presently claimed subject 
matter can solve the edge problem described in Haala without the 
need for 2D CAD (Computer Aided Design) or GIS (Geo Information 
System) data representing ground plans for the structures being 
modeled. 

The art of record fails to teach or suggest the features 
now claimed. Thus, claims 2, 3, 18, 41, and 45-49 should be 
allowable . 

Claims 24-28, 30-36, 38, and 42-44 stand rejected under 35 
U.S.C. 103(a) as allegedly being unpatentable over Kumar in view 
Arpa. Claims 31 and 39 stand rejected under 35 U.S.C. 103(a) as 
allegedly being unpatentable over Arpa in view of Kumar as 
applied to claim 24 above, and further in view of Haala. These 
contentions are respectfully traversed. 

Claims 24, 27, 28, 32, 35, 36, and 42-44 have been 
cancelled by this amendment without prejudice. The remaining 
claims 25, 26, 30, 31, 33, and 34 now depend from an allowable 
base claim. Thus, the rejections of claims 24-28, 30-36, 38, 
39, and 42-44 have been obviated by the present amendment. 

It is believed that all of the pending claims have been 
addressed. However, the absence of a reply to a specific issue 
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or comment does not signify agreement with or concession of that 
issue or comment. Because the arguments made above may not be 
exhaustive, there may be reasons for patentability of any or all 
pending claims (or other claims) that have not been expressed. 
Finally, nothing in this paper should be construed as an intent 
to concede any issue with regard to any claim, except as 
specifically stated in this paper, and the amendment of any 
claim does not necessarily signify concession of unpatentability 
of the claim prior to its amendment. 

•It is respectfully suggested for all of these reasons, that 
the current rejections are overcome, that none of the cited art 
teaches or suggests the features which are claimed, and 
therefore that all of these claims should be in condition for 
allowance. A formal notice of allowance is thus respectfully 
requested. 



32 



Serial No. : 10/676,377 



Attorney's Docket No .: 06666/156001 



Enclosed is a $300 check for excess claim fees. Please 
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